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ABSTRACT 



This report evaluates a project undertaken in the Czech 
Republic to create lesson scenarios to revise the existing social studies 
curricular framework for the third form of secondary school (ages 17-18) with 
special reference to the overarching objectives for civic education reform 
started in 1989. These objectives included the elimination of 
Marxist-Leninist perspectives in the curriculum; a renewed study of Czech 
history, culture, heritage, and geography; and a pedagogical shift from 
transmitting information to passive students to the prompting of inquiry and 
active learning. The evaluator's activities included a presentation about 
evaluation to the lesson design team of Czechs, development of draft versions 
of instruments to gather feedback about the lessons, attendance at meetings 
at which field testing was planned, observation of training for 
computer-assisted simulation, attendance at other relevant meetings, and 
preparation of the report. Three assumptions and decisions framed the 
evaluation: (1) the focus should be almost exclusively on the lessons being 

developed, their quality, and how they were being received by teachers and 
students during the field test period; (2) use of several methods of data 
collection and sources of data was the only way to assure that the necessary 
depth of information about the lessons could be obtained; and (3) the "center 
of gravity" for both collecting and analyzing that data should be in the 
Czech Republic and not at the Universities of Maryland or Iowa. The report 
outlines the project in detail and summarizes the strengths and weaknesses in 
its planning and design, the lessons, training and capacity building, and 
communication and motivation. Contains a table and 8 notes. Appendix 1 
features lesson topics, types of student participation, and teacher and 
student feedback. Appendix 2 contains notes on interviewing teachers. (BT) 



******************************************************************************** 



* 

* 



Reproductions supplied by EDRS are the best that can be made * 

from the original document . * 



******************************************************************************** 




ED 431 693 



Civic Education in the Czech Republic: Curriculum Reform for 
Democratic Citizenship. Evaluator’s Report. 



by Judith Tomey-Purta 



o 

oo 

oo 




o 

CO 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

Judith 

Tbr n — 

TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 

1 



U S DEPARTMENT OF EDUCATION 
mice of Educational Research and Improvement 
UCATlONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

This document has been reproduced as 
received from the person or organization 
originating it. 

Minor changes have been made to 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




AVAILABLE 



Evaluator's Report 

Civic Education in the Czech Republic: 

Curriculum Reform for Democratic Citizenship 

Judith Tomey-Purta, Ph.D. 1 
Professor of Human Development 
University of Maryland 
College Park, MD 20742 

Submitted January 1999 

Overview of the Project 

This project's aim was to create lesson scenarios in order to revise the existing social studies 
curricular framework for the third form of secondary school (ages 17 and 18) in the Czech 
Republic with special reference to the overarching objectives for civic education reform 
started in 1989. These objectives included the elimination of Marxist-Leninist perspectives 
in the curriculum; a renewed study of Czech history, culture, heritage, and geography; and 
a pedagogical shift from transmitting information to passive students to the prompting of 
inquiry and active learning. The project was conducted as a joint effort of the pedagogical 
faculty at Charles University in Prague, the Czech Republic and the University of Iowa 
College of Education, funded by the United State Information Agency and began in mid- 
1995. The major written product of the collaboration was a book of sample lessons on 21 
topics exemplifying the curricular objectives outlined above and an accompanying teacher's 
manual. For other major components of the project, see the ERIC Digest EDO-SO-97-5 
prepared by Gregory Hamot, Co-Director of the Project and other project documents. 

Overview of the Evaluation 

The responsibility of the evaluator was to provide commentary throughout the project as 
formative evaluation and summative evaluation of the lessons and manual (and to the extent 
possible at a distance, the process of development, try-out, and revision). I as the evaluator 

• made a presentation about evaluation to the lesson design team of Czechs in Iowa 
City in October 1996; 

• developed draft versions of instruments to gather feedback about the lessons; 



'The major collaborator on this report is Barbara Malak-Minkiewicz, Ph.D., Senior Program 
Officer, IEA Headquarters, Amsterdam. Also of great assistance in the data collection and 
analysis in Prague were Pavla Palechova, Radmila Dostalova, Bohuslav Dvorak, and Vera 
Dvorak. Special assistance at the University of Iowa, in addition to Greg Hamot and Peter 
Hlebowitsh, the Co-Directors, was given by Joe Bishop. 



• attended and made a presentation at meetings at which field testing was planned and 
the feedback sheets for teachers and students to use in evaluating the lessons were 
finalized in Prague in January 1997; 

• facilitated the attendance of two Czech participants at a meeting with University of 
Maryland personnel providing training for the ICONS computer-assisted 
international simulation in Berlin in February 1997 (with the hope of providing 
narrative material from students for the evaluation as well as introducing a 
computer-assisted mode of communication); 

• attended a meeting in Iowa City in July 1 997 at which the data were examined by 
the bi-national team; 

• attended and made a presentation at the conference at which the book of revised 
lessons was released to a broader group of teachers in the Czech Republic in August 
1998 

• in advance of that meeting provided to all of those from the U S. who planned to 
attend a brief written report of the data collected including suggestions about ways 
to tailor their presentations to the perspectives of the Czech educators as it had 
emerged from the lesson evaluations. 

• prepared the summary table in Appendix 1 and the text of this report 

Barbara Malak-Minkiewicz, a social psychologist who was involved as an expert of the 
Polish Ministry of Education in some civic education reform projects in Poland in the early 
1990s, also attended the meetings in January 1997, July 1997 and August 1998 and 
reviewed this report, prepared one section of it, and prepared suggestions for interviewing 
teachers (Appendix 2). 

Three assumptions and decisions framed this evaluation, assumptions that differ somewhat 
from those which frame many evaluations. 

The first assumption was that the evaluation should be focused almost exclusively on the 
lessons being developed, their quality and how they were being received by teachers and 
students during the field test period. The purpose of teaching the lessons in the spring of 
1 997 was to suggest improvements to be made and material to be added. Each teacher 
who was participating in piloting taught only a small portion of the 20 lessons that had been 
developed. Thus any given class of students was exposed to only a partial segment of the 
curriculum, and different classes had different lessons. Time was limited, so that some 
classes had a few as three or four hours of these lessons. Teachers were encouraged to 
make selections from the lesson materials and to adapt them to student interests and needs. 
For all of these reasons it was not appropriate to conduct any kind of evaluation of student 
outcomes (knowledge or attitudes). Evaluation of student outcomes is appropriate only 
when there is an agreed upon program of lessons which each participating teacher and class 
uses and when it is possible to document a similar implementation of the program of lesson- 
scenarios by different teachers. Ideally in such outcomes evaluation the classes to 
participate have also been randomly assigned. Even the minimum conditions for an 
outcomes evaluation can be fulfilled only after the field test period has been completed and 
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a full and relatively final curricular program is implemented. 

The second assumption was that using several methods of data collection and sources of 
data was the only way to assure that the necessary depth of information about the lessons 
could be obtained. Further, evaluation should be an integral part of the entire project and 
not confined to a period at the end. Data were collected from teachers and from a small 
number of students (2 to 5 per classroom per lesson). These data were both quantitative 
(in the form of ratings) and qualitative (in the form of written answers to questions about 
the lesson, as well as in group and individual interviews). Some of these data (e.g., teacher 
and student ratings) could be examined separately by lesson or group of lessons. In addition 
to providing data for the evaluation, asking students as well as teachers for their opinions 
was a way of communicating that this was a new model for designing curriculum. The 
teachers were treated as experts from whom the evaluators and project designers were 
seeking advice in order to improve the lessons. 

The third assumption was that the "center of gravity" for both collecting and analyzing that 
data should be in the Czech Republic and not at the University of Maryland or the 
University of Iowa. This created a very heavy workload for the Czechs, since materials to 
be used in data collection had to be translated from English into Czech, then the responses 
had to be translated from Czech back to English. If this report and its appendices are to be 
fully utilized, it will have to be translated again back into Czech. The side benefit of 
decentralizing the evaluation in this way is that there is now a cadre of individual specialists 
in civic education in the Czech Republic who are prepared to conduct their own evaluations 
of similar projects. 2 If one believes that the sustainability of a project such as this depends 
upon a continually expanding group of teachers and specialists who are engaging in an 
ongoing dialogue about the reform of civics teaching, this approach to evaluation is clearly 
preferable to other models. 

Details of the Evaluation in the Czech Republic 

Draft versions of the rating sheets to be used by teachers and students for each lesson were 
prepared at the University of Maryland based on an examination of the objectives of the 
project and on previous experience. The student instrument was one page per lesson and 
included ratings of how interested the student had been and how much they participated. 
The teacher instrument was about 3 page long and included ratings and 17 open ended 
questions (see details in next section). 

During a project meeting in Prague in January 1997 a group of about ten met to finalize this 
instrument, including those who had been asked to translate the instruments. That meeting, 
in which the Czechs suggested many changes to the wording (either because the original 



2 See also Judith Tomey-Purta (1998), Evaluating programs designed to teach international 
content and negotiation skills. International Negotiation, 3, pp. 77-97. 
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formulation was difficult to translate or because it seemed an inappropriate or trivial 
question) was another example of the importance of the Czech perspective in the overall 
project design. Other sessions at this meeting provided training in the evaluation methods. 

Following that meeting, the evaluator laid out the expectations for each Czech participant. 



Each of the 9 pilot teachers was expected to fill out a feedback sheet for each lesson 
taught and give these to researchers/ coordinators being sure lesson number and title 
were on the sheet. They were also asked to choose 3 or 4 students during each class 
period who would fill out the student feedback sheets (being sure not to pick the 
same students each time). They were also expected to participate in an interview 
at the end of the period and choose students to participate in a group interview 
(focus group). 

Members of the project team were designated as "researcher /coordinators" and had 
special responsibilities. They were to explain this procedure to teachers and 
encourage them to fill out the sheets, reassuring both groups that the purpose of 
their information is to improve the lessons and not to evaluate teachers. They were 
also to collect the sheets and conduct the interviews. The senior researcher 
arranged for the translation of the comments, then condensed them on summary 
sheets which were presented to the evaluator and to the group in Iowa City in July 
1997. The comments in the original Czech were sent to the coordinator who was 
making revisions in the lessons before their publication. 

The implementation of the field test and evaluation took place as follows. Every lesson was 
taught at least once and in most cases twice. There was considerable variation in the way 
that the same lesson was taught by different teachers (for example merging some sections 
or stressing facts rather than discussion). Researchers experienced a heavy workload and 
too little time for making summaries. It seemed that the teachers accepted evaluation as 
part of lesson development, and that their comments could for the most part be integrated 
into the revision. Special attention was paid to revising lessons where more than one 
teacher saw problems. Because of close contact between the authors of lessons, researchers 
and teachers it was possible to actually check out some suggested revisions, e.g., could this 
be changed in this way to meet your criticism? Ratings were sometimes used as a basis for 
discussion with teachers. Often teachers said they wanted more concrete examples, but it 
was difficult to add as many pages as would have been necessary to do this. Some teachers 
thought goals were too numerous and detailed, and many of their suggestions for 
simplification were taken. It was clear that the teachers needed more in-service training not 
only in content but in pedagogical science and methods. They often did not know how to 
evaluate student interest or were unused to selecting some material and leaving out other 
material. 

The feedback sheets were collected, tabulated and translated for entry onto summary sheets. 

4 




6 



The next section represents the evaluator's summary in text, andAppendix 1 represents a 
review of both the content of specific lessons (in the first two columns, prepared by two of 
the Czechs responsible for the lesson construction) and the evaluator's summary of material 
on the translated rating sheets filled out by both students and by teachers (in the third 
column). This table thus presents both the lessons' contents and methods and the way they 
were evaluated by the teachers who taught them and the students in those classes. 

In addition 6 teachers were interviewed, 5 face-to-face and 1 in written form. It was 
reported that most interviews were conducted with a "good atmosphere." One refused, one 
said there was time only for informal comments, and one could not be scheduled. There was 
considerable variation in the teachers -- several felt very competent (even inspired) and 
reported having exceptional results; one or two were enthusiastic but had difficulty 
accommodating to the new types of teaching; one or two were not positive about the 
experience feeling that it required too much work. 

There were also focus groups with student in the six of the schools (same as those where 
interviews were conducted). One teacher conducted focus groups on her own. 

It was originally hoped that ICONS Computer-Assisted International Simulation at the 
University of Maryland Department of Government and Politics could be integrated as a 
source of both new modes of active student participation and of data from students for to 
be used as illustrations in the evaluation. With that in mind, two Czech project participants 
were supported with funds from the evaluation to attend a training meeting in Berlin. Two 
schools volunteered to participate, but one had such serious technical problems that it was 
impossible for them to be fully integrated. One school did participate in a Designing 
Democracy cross-national on-line conference in which they were communicating with 
students from the U.S. and Canada. The following interchange included a message from the 
Czech students: 

From Simulation Coordinator: (April 21, 1997, 12:46) 

Here's a hypothetical situation. Suppose a military leader in Libroslavia (a mythical 
country just becoming independent) offered to take control of the government for 
several years until Libroslavia could become stable economically and political. 
Would it be a good idea to let him take over? Why or why not? 

From team of students in USA: (12:51) 

No, the idea that military leader should take over Libroslavia would not be 
beneficial. History has proven that there are a lot of military leaders that go in with 
that belief and end up as dictators. This is not true for all, but in most cases the 
situation does apply. 

From team of students in Canada: (12:53) 

We have would have no problem with a military leader as long as he is actively 
democratic. He must be able to unite ethnic cultures of the nation. We would also 
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would like to remind the USA that some of their great leaders were Military 
Leaders. 

From the team of students in Czech Republic: (12:55) 

The problem is that the military leader would tend to keep the power as long as 
possible. Nobody can leam democracy in dictatorship. We suggest Libroslavia to 
accept advisors from democratic countries, but the people of Libroslavia have to 
govern their country by themselves. 

The Czech students are to be commended for a well thought out message, coherent and 
entered into the Web system promptly. As computers become a more widespread part of 
the technology available in Czech schools we can imagine that possibilities for this 
participation will increase. The ICONS project, which is ongoing at the University of 
Maryland, has issued an open invitation for the future for free participation in any of its 
simulation exercises by any interested Czech schools where students can communicate in 
English and where there is access to the World Wide Web. 3 

The remainder of the report deals with data collected in a systematic fashion from the 
participating field test sites. 

Data from Feedback Sheets: Quantitative Ratings and Qualitative Comments 

Feedback sheets, containing spaces for both ratings and for open comments, were filled out 
by teachers and students at the conclusion of each lesson. They were summarized and then 
translated into English (thanks to the Czech collaborators). The original sheets (in Czech) 
and summaries were provided to those revising the curriculum for the purpose of making 
improvements in the text (and making deletions and/or additions of new materials). 

From Students' Ratings: 

Students (usually 3-4 per class) filled out two scales for each lesson, one rating on a 4 point 
scale how actively they were involved during the lesson and the other how interested they 
were during the lesson. Student ratings within a class were averaged. Taken across the 72 
times that lessons were taught, these ratings were correlated .68, meaning that those who 
reported that they were actively involved were also more interested. 

The 21 lessons were placed into eight categories. Table 1 gives information about the types 
of lessons and mean ratings of student interest and activity. On a 4 point scale the mean 
(average) across lessons for ratings of Activity was between 2.40 and 2.85, and for Interest 
between 2.76 and 3.53, tabulated by lesson category. The students expressed the highest 



3 The contact person at the University of Maryland who worked with the Czech educators and 
could be contacted for materials about future participation is Beth Blake, bblake@bss2.umd.edu. 
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amount of interest and the greatest amount of participation for the lessons categorized under 
Social Problems and under Political Theory, the lowest for lessons categorized under 
Domestic Institutions and under World Issues and Institutions (and to some extent under 
Law). Other lesson categories received moderate ratings. 

Some contrasts with the content of the civics curriculum in the US. may be of interest. The 
teaching of Institutions seems quite similar between the countries in its basic organization 
(though the specific details of content differ). The same is true for Civic Action and Law. 
The greatest contrast in content was in the Political Theory category. The Czech materials 
were oriented toward political philosophy and the ideology of democracy much more than 
would be the case in the U S. 

From Students' Comments: 

Students also answered open-ended questions: 

What did you learn today in class? 

What class activities helped you to learn today? 

What did you not understand today? 

What would you recommend to authors to change so that the student like 
you would gain as much as possible? 

There was a good match between what teachers were teaching and what students said they 
learned (see open ended comments cited later). Many wanted even more opportunities for 
discussion and concrete materials/visual aids or more about contemporary 
problems. Students expressed difficulty in understanding many of the same issues that 
students all over the world have trouble grasping, for example concepts such as recession, 
inflation, or embargoes. Other comments reflect a common attitude of puzzlement about 
why countries engage in aggressive foreign policies, why there is so much hate among 
political parties, why censorship exists, and why state and people don't do more about 
violence. 

Students' comments were generally positive about the lessons, noting especially that they 
learned how to express or assert their opinions and listen to others, how to face up to 
problems, and how think carefully and deeply about issues. 



Table 1 

Student Activity and Interest Ratings by Lesson Category 
(Means Across Classes/Lessons 
and Rank of Means within Column in [ ]) 



Category/Topic 


Lesson Numbers 


Activity 


Interest 


I. Political 
Theory 

(polit. philosophy, 
democratic ideology) 


1-3 


2.85 [2] 


3.26 [2] 


II. The State 
(history, contemp- 
orary evaluation) 


4-6 


2.74 [4] 


3.20 [3] 


III. Domestic 
Institutions 
(Constitution, 
executive, legislative, 
judicial, municipal, 
elections) 


7-10 

part of 1 3 


2.58 [7] 


2.82 [7] 


IV. Civic Action 
(Skills and behavior: 
letter writing, NGO's, 
public meetings, parties) 


11-13 


2.68 [5] 


2.99 [5] 


V. Law 

(Constitutionally based) 


14-15 


2.76 [3] 


2.76 [8] 


VI. Economics 
(Free market, money 
and banking) 


16-17 


2.60 [6] 


3.09 [4] 


VII. Social Problems 
(AIDS and drugs) 


18 


2.86 [1] 


3.53 [1] 


VIII. World Issues 
and Institutions 
(Ecology, terrorism, 
UN, NATO, EU) 


19-21 


2.40 [8] 


2.93 [6] 
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From Teachers' Ratings : 



The teachers were asked to fill out 1 1 rating scales (4-point rating) for each teaching of a 
lesson: 



This lesson leads the students to: 
participate actively in the class 
cooperate in groups 
gain citizenship knowledge 
acquire active citizenship skills 
develop citizenship attitudes 

Overall this lesson: 

gave me an opportunity to teach a subject matter 
important for its content 
allowed students to discuss their opinions 
allowed students to draw their own conclusions 
had about the right amount of material 
objectives of the lesson were met 
was interesting to my students. 

The means across ratings were in ranges similar to those for students, ranging from 2.75 
(extent to which the lesson allowed teaching of active citizenship skills) to 3 .48 (perceived 
student interest). The modal (most frequent) response was a 3 (somewhat) for all 11 
ratings. 

In addition to making the ratings, teachers were asked several pages of open-ended queries 
about each lesson: 

The following subject matter was left out. 

The following activities were left out. 

The following subject matter was added. 

The following activities were added. 

The following subject matter was revised in this way. 

The following activities were reorganized in this way. 

Other changes I made. 

What are the strengths of this lesson from the contents' point of view? 

What were some positive things that happened during the class time or activities that 
communicated the point of the lesson well to students? 

What are the strengths of the lesson from the goals' point of view? 

What are the weaknesses of the lesson from the contents' point of view? 

Did anything appear in the lesson that students did not understand or did not get? 
Were there any negative things that happened during the class time (e g, attitudes 
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of students?) 

How could the material for this lesson be improved? 

Should anything still be left out? 

Should anything still be added? 

Other changes that should be made? 

The patterns of correlations within the teacher data give us some insights into the 
implementation of these lessons in the classroom (although these must be viewed as 
suggestive, since there were 9 teachers and 72 lessons). First, the teachers were not very 
accurate informants about student interest or how involved the students were in a particular 
lesson. The correlations between averaged students' ratings and teachers' ratings of 
perceived interest by lesson were low and not statistically significant. It is important to gain 
both perspectives when getting feedback on implementation in the classroom. 

One question given to the teachers, the extent to which the lesson led the students to 
"cooperate in groups," appears to have been misleading. This rating showed few 
correlations with other ratings, and the comments suggest that in Czech classrooms fruitful 
discussion takes place not only when the teacher places students into small groups asked to 
cooperate but also in larger group settings. "Cooperative groups" and "cooperative 
learning" are terms much more frequently used in American pedagogical discourse. In 
Czech classrooms having discussions where students can voice their opinions is important, 
but the size of the group or its assigned task as a "cooperative group" is not so important. 
In the Czech classroom the pertinent contrast seems to be between an exchange of opinions 
in a discussion and lecture or individual reading/writing. 

Substantively it appears that fostering discussion as a part of the training provided to Czech 
teachers should consist not only of models based on American cooperative learning 
methods, but should encompass other methods of ensuring a climate for free exchange of 
opinions within the classroom whether in a small group or a larger classroom group. 

Likewise the question about whether the lesson "had about the right amount of material" did 
not correlate well with other teacher ratings (e g. student activity, interest, whether 
knowledge, attitudes or skills were gained). This is understandable if we consider that for 
about 90% of the lessons teachers reported making some kind of alteration in the material 
(deletions, additions, or both). It is clear that the large majority of teachers tailored their 
teaching to the students in their classrooms (including vocational students). They reported 
adding examples from real life, newspaper articles relating to the content, charts making 
comparisons (for example of markets at the local, national and international level) or giving 
statistics (about vandalism), explanations of new terms (especially in economics), new 
student activities (such as an assignment to formulate a meeting agenda), and materials to 
tailor the lesson to the Czech situation (such as lists of Czech achievements in various 
fields). Again from these qualitative findings it appears that these highly professional 
teachers made judgments so that the amount of material covered in the teaching was 
appropriate. When there was too much material, they cut some sections or left activities 
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out. The success of teaching different lessons did not seem to be influenced by the fact that 
some lesson plans originally had more material to be covered than others. When the next 
group of teachers gets the revised lessons, however, it will probably be important that the 
instructions make clear that the teacher can choose from materials suggested and/or add 
materials. Teachers may also need training in how to make these decisions. 

There are quite a number of clues to the ways in which the teachers creatively worked with 
the materials to foster an appropriate classroom climate and coherent subject-matter content 
learning for the students within the correlations between ratings. 

Let us look at the correlations between perceptions that the lesson led students to: "gain 
citizenship knowledge," "acquire active citizenship skills", and "develop citizenship 
attitudes." The correlations here are among the highest in the matrix (.82 between 
knowledge and attitudes, .77 between skills and attitudes, and .65 between knowledge and 
skills). Most teachers thought that those lessons which succeeded in one respect succeeded 
in other respects (and those lessons that were less successful in one respect were also less 
successful in others). 

Correlations between the "objectives of the lesson were met" and the other rating scales can 
indicate what kind of lessons were most successful in the teachers' views. Gaining 
knowledge is the most highly correlated to this rating of the meeting of objectives (r=.74), 
with a slightly lower correlation between objectives-achievement and "developing attitudes" 
(r=62) and a much lower correlation with "acquiring active citizenship skills" (r=. 49). One 
gets a sense that it was somewhat unclear to these teachers what "active citizenship skills" 
meant and how the lessons were addressing them. The suggestion here is similar to that 
offered regarding cooperative groups. Training programs should consider developing a list 
of skills relating to citizenship which includes but is not limited to American formulations 
of active citizenship skills. For example, there may be skills of analysis of text, skills of 
listening, and skills of reflective thought that are important within the Czech context which 
should be represented. 

Next, let us examine the correlations (r's) within the set of items that dealt with student 
participation and involvement. Lessons in which students are "led to discuss their opinions" 
are also lessons in which students are "allowed to draw their own conclusions" (r=.71). 
Further, lessons in which students "participate actively in the classroom" are those in which 
the lesson was "interesting to students (r=.65) and students were "led to discuss their 
opinions" (r=.64). The teachers clearly seemed to have gotten the message that discussion 
was important and felt that, at least in some classes, they were able to get the students into 
this mode of learning and classroom process. 

Further evidence of that comes from the following. Teachers tended to perceive that the 
"objectives of the lessons were met" very well in the following situations: 

"lesson gave me an opportunity to teach a subject matter" (r=,77), 
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"lesson was interesting to my students" (r=.74), 

"lesson led student to gain citizenship knowledge" (r=.72), 

"lesson led students to participate actively in the classroom" (r=.69), 

"lesson led students to develop citizenship attitudes" (r=.62). 

"lesson allowed students to discuss their opinions" (r=.54) 

"lesson allowed students to draw their own conclusions." (r=.52) 

As previously noted the correlations with "led students to develop active citizenship skills," 
and "had about the right amount of material" were lower (r's of .49 and .39 respectively). 
The correlation with "allowed students to cooperate in groups" was .06 and not statistically 
significant. This suggest that the teachers felt that the pedagogy and the content worked 
together to provide lessons which met their objectives. Discussion and participation were 
important parts of the lessons, which were also shaped by the presentation of important 
subject matter content. 

From Teachers' Comments: 

Further insights about the process of the field testing of lessons can be illustrated from some 
comments about individual lessons made by teachers (presented first) and students 
(presented in brackets). 

The lesson shows the necessity of encountering different opinions in 
democratic society. [Student comment: learned from research and 
discussion] (Concept of democracy lesson) 

I liked the lively reworking of history and that students came to definitions 
of patriotism on their own. [Student comments: learned from discussion ] 

(Nation and state in Czech history lesson) 

I liked the way practical tasks supported previous more theoretical 
knowledge; students by determining which right is most important 
understood interconnections and importance of rights for freedom of man; 
students may get less amount of knowledge but those facts that are gained 
are deeper and more pertinent because they are based on own experience 
and emotion. [Student comments: learned to work collectively in a group 
and that all rights are interconnected; use this style of lesson more often; 
learned fast decision making and to listen to opinions of others ] (Human 
rights lesson) 

Students formed own meaning of importance of elections by participating in 
one. [Student comment; learned about system and how to vote.] (Elections 
lesson) 

They wrote a petition; strong lesson because it was concrete, best to this 
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point. [Student comment: learned about modes of protest or complaint 
about environmental problems; rights to join organizations on a voluntary 
basis; learned about writing a petition.] (Civic community lesson) 

I made a discussion of being a good listener a starting point for discussing 
free speech, tolerance, civil rights; and got student self evaluations. [Student 
comments: learned to analyze my own qualities and listen to others.] 

(Citizen and the state lesson) 

I added material about styles of meeting, agenda, report. [Student 
comments: learned how to organize a public meeting and get information 
about our community; rights to decide about public affairs; preparing to 
contribute to discussion] (Citizen and the state lesson) 

Students may get less quantity of knowledge, but what they acquire is 
deeper and will last longer because they made their own conclusions. 
[Students comments: learned today by listening to others' opinions and 
finding compromises in solving problems ] (Municipality and the individual 
lesson) 

Goals were fulfilled; need more examples of function of currency. [Student 
comment: I overtaxed my brain by making it really think, it was an 
exceptional lesson in which I proved to myself that I knew something about 
the economy ] (Economy and ways to prosperity lesson) 

Please bear in mind that these comments have been summarized and excerpted and that they 
were made as feedback to the lessons before they were revised. Appendix 1 gives much 
more detail and is organized by lesson. 

Data from Focus Groups of Students 

In the focus group when asked whether their learning took place primarily because of 
changes in the content of instruction, the process, or both the large majority of students said 
that it was because of the process. When asked which of their concepts changed the most 
toward a better understanding. Freedom and Responsibility were reported to be the most 
influenced. Most students reported little change in concepts of Minorities, Privacy, and 
Authority. Both these findings are understandable since there were several lessons focussing 
on freedom and responsibility and very few on minorities, privacy, or authority as general 
concepts. 

Several students in the focus groups noted that many discussions took place as a whole class 
rather than in small groups. Among the topics which they singled out as receiving special 
attention were: how to help the Republic politically and economically (by buying Czech 
goods), respecting the law even if they didn't agree with it and respecting others as part of 
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freedom to express opinions; and the important role of participating in decision making in 
school and other contexts. There was a comment that "the history of political science did 
not interest us, too many facts and too few sources or references." Several wanted more 
practical examples (echoing the written responses), discussion, student participation, and 
current political events. Several groups noted that there was insufficient time, and that they 
would like to experience this type of instruction more often. A concern which surfaced from 
a number of students was how they would be graded when methods were used which are 
new or do not lend themselves to factual testing. 

Data from Interviews with Teacher s 4 

Although the Czech- American project in many ways constituted a replication (for the Czech 
Republic) of the cooperative project between the Polish Ministry of Education and the 
Mershon Center of the Ohio State University 5 one important step forward was made. The 
evaluation of the lesson scenarios was based on their testing by Czech teachers. This is 
innovative, since in most previous projects (including the Polish one) the evaluation had 
been limited to the opinion of American experts (under the assumption that the materials and 
approaches might be too new and difficult for teachers in post-Communist countries 
undergoing extensive educational reforms). Accepting teachers as experts for evaluating 
lessons has many advantages, since both weaknesses in the scenarios and teacher difficulties 
(especially those reflecting deficits in teacher preparation) can be discovered in this way. 
(Even the best lesson proposal is not good enough if Czech teachers are not able to carry 
it out.) 

One of the methods for collecting data from teachers was the interview. Two obstacles had 
to be overcome, first the lack of tradition in recognizing teachers’ expertise in this context 
and, second, the fact that the Czech researchers available for this project lacked experience 
in interviewing. 

Detailed guidelines about how to interview teachers were developed (see Appendix 2) and 
discussed with researchers during the meeting in January 1997 and through email. They 
considered the instruction useful and reported that it contributed to the collection of useful 
data for revision and improvement of lesson scenarios. The main conclusions from the 
interviews of six teachers are presented below. (In additions to civics, three also taught 
Czech literature or history, and one was a foreign language teacher). 

All the interviewees evaluated the lesson-scenarios positively. They did not have time to 
familiarize themselves with the full set of scenarios, but two major strengths they found in 



4 This section relies on material prepared by Barbara Malak-Minkiewicz. 

5 See Richard Remy & J. Strzemieczny (eds.), (1996), Civic Education for Democracy, 
Lessons from Poland, Washington, D.C.: National Council for the Social Studies and ERIC. 
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those they did review were that they proposed active teaching methods, including useful 
examples, and that they included helpful resource material. 

Active teaching methods: 

Teachers view this kind of method as an effective didactic tool and an important element to 
add to their own didactic equipment. The wish to learn more about these methods was part 
of their motivation to partcipate in the project, and they were planning to make use of them 
in other subject lessons. In the opinion of these teachers, active teaching methods are also 
useful specifically in civic education: contributing to the development of critical thinking, 
the ability to formulate and express opinions, tolerance for diversity, and work in groups. 
They also identified several problems related to these methods for both students and 
teachers. First, they may weaken discipline in that some students may become noisy, while 
others may refuse to cooperate. Teachers believed that this problem is especially important 
in post-Communist countries with their overcrowded classrooms and the rigid (almost 
military) atmosphere which still persists in many schools. As one of them said, “it is not 
only a matter of teachers; the student need time to adjust to a new method too.” Active 
methods were also perceived as much more demanding in terms of preparation for the 
lesson, its organization and timing. One teacher said, “during this period (testing of 
scenarios) I didn’t have time to talk to my colleagues; they could see me only in the library 
or at the copying machine.” 

Some recommendations that result from the above go beyond this particular project. There 
is a need to include active teaching methods into the teacher training, including that for in- 
service teachers. These methods should also be recommended as valuable in teaching 
various school subjects, not just civics. Such changes might contribute to the improvement 
of the general atmosphere or climate in school. Policy makers should become aware that 
teachers need additional, specific incentives to work with active methods, since they are 
much more challenging, demanding, and time-consuming. Teachers themselves see this 
problem clearly. 

Source Materials: 

The interviewed teachers were enthusiastic to find various kinds of authentic source 
materials that could be used during the lessons as attachments to the scenarios. These 
materials helped them link discussions to reality. It was quite a new experience for them; 
in Communist schools they either had to use textbooks or search for original materials 
themselves (and only the highly motivated did so). 

There were also some doubts, however, concerning how these materials could be used. One 
teacher said, “They gave me the American Constitution, the English Constitution. . . and I 
only wanted kids to understand the Czech Constitution.” Some teachers suggested that in 
the lesson-scenarios it should be clearly stated what should be obligatory and what should 
be optional. These concerns are only partly the result of lack of experience with using 
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source materials in class. During the interviews, teachers were trying to put tested lessons 
in the famework of the civics curriculum in their country, reaching conclusions like these: 
“the themes are the same or similar but in the real curriculum there are ten times as much 
topics,” or “I do not see how these scenarios could be integrated with the existing courses.” 
Teachers seem to suggest that, if programs such as these are to be used broadly, there is a 
need for specific instruction on how such a program could be integrated into an existing 
curriculum. This leads to an important recommendation for this and other projects: it should 
be clear from the beginning whether materials being prepared are meant to replace some 
of the existing programs, serve as an optional alternative for them, or some combination. 

Summary of Evaluation 

In this section the strengths and weaknesses of each of the major parts of the project will 
be assessed from the Evaluator’s point of view. 

Evaluation of the Planning and Design of the Project: 

Strengths: 

Although American models were used and American experts consulted, there was a distinct 
effort to attend to the issues which the Czech participants thought appropriate and 
approaches which had been honed by experience in a previous project in Poland (see 
comments above). Lessons learned from other projects contributed to an excellent 
experience in Iowa City for the authors, in which they were exposed not only to American 
ideas but to some models developed elsewhere in post-Communist countries and were given 
a wide range of models upon which they might build. 

The interweaving of evaluation throughout the implementation process and the placing of 
the center of gravity for the evaluation in the Czech Republic (especially the consultation 
of teachers about the lessons) was an important and positive feature of the project. This is 
one of the major ways in which aspects of sustainability were gained in the project; in fact 
two participants spoke especially about how much they had learned about evaluation 
methodology from taking these responsibilities. 

Weaknesses: 

Almost all projects aim at too many changes that are expected to take place simultaneously 
within a social context which is itself changing, making it difficult to designate benchmarks 
and achievements toward them. In this project also, there were many goals, and they were 
quite far reaching. 

Although one might prescribe less lofty goals for future projects, it is also the case that those 
who work on these projects usually do so for idealistic reasons. Having lofty goals (fully 
understanding that not all of them can be achieved) is important to them as part of the 
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project design. A more limited and less idealistic plan would be less attractive to those 
required to carry it out (and might not even achieve the minimum levels of support from 
agencies or foundations, who themselves may be looking for great achievements while 
underestimating the time necessary to achieve reforms). 

Evaluation of Lessons. Development Process, and Trials : 

Strengths: 

The project seemed to be successful in developing a set of lesson scenarios which was well 
received by teachers and students. It also was successful in promoting a sense of excitement 
and commitment to the project among most of the participating teachers and opening a 
dialogue about changes and the importance of incorporating new pedagogical methods and 
new content. Most participants understood what was expected and wanted more 
opportunities to explore and practice these methods. Teachers also commented that they 
felt that these methods were successful in engaging the students in experiences with 
considerable depth. It also appears that teachers' and students' comments were reflected in 
the redesign of lesson materials. 

Weaknesses: 

Extreme time pressure in the Spring of 1997, coupled with shifts in personnel and their 
assignments in the Czech Republic, resulted in some confusion and in fewer teachers than 
planned being able to pilot lessons and fewer opportunities for in-depth examination of the 
experience. 

The costs and time required for translation of materials was seriously underestimated. At 
one point it was planned to translate the lessons back into English, but it became obvious 
that was prohibitively expensive (and thus the lesson summaries to be found in Appendix 
1 ). 

The absence of computers with World Wide Web connections in many of the schools, 
coupled with lack of confidence on the part of teachers in using such methods meant that 
the ICONS part of the project could not be fully implemented. However, two individuals 
received training in the method and may be able to assist in implementation in the future 
when technical resources become available. 

Evaluation of Training and Capacity Building : 

Strengths: 

The project was successful in building capacity and involving individuals at all levels of the 
educational system in the Czech Republic -- personnel of a governmental institute, 
curriculum specialists, teachers, and students. Nearly all of these individuals reported 
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learning from the experience (reflected in personal comments as well as the more formal 
interviews). Areas where further training is needed, such as pedagogical methods, were also 
identified. 

Weaknesses: 

There were some concepts such as "active citizenship" and "cooperative groups" which 
were hard to translate across cultures. It was not merely an issue of linguistic equivalence 
but of the need for further dialogue with Czech participants concerning ways to get the core 
ideas behind these notions into implementable form and ways to formulate training programs 
to address them. 

Variations across teachers in how lessons are taught and how material is used are inevitable 
(and not a bad thing); some of the newer pedagogical methods need to be addressed in 
training courses which are formulated with the same attention to the perspectives of 
different groups as the lessons received. 

Evaluation of Communication and Motivation: 

Strengths: 

The level of motivation was very high. Furthermore, an important message was conveyed 
by asking students and teachers for their opinions about lessons, and it allowed the 
adaptation and revision of lessons to meet not only overall goals of the educational system 
but also the specific needs of teachers and students (including those in vocational tracks). 

All the interviewed teachers said that they were very interested and excited when preparing 
and conducting lessons. They also reported getting new ideas about practical exercises for 
students as well as new and interesting source materials. If this set of lesson-scenarios could 
help to revitalize teachers’ creativity and give them more satisfaction in their work, this 
would in itself be a big achievement for the project, because such opportunities were so 
limited during the period of Communist control and have not expanded quickly enough in 
the period since. 

Weaknesses: 

There were communication breakdowns from time to time, resulting from a variety of 
factors including distance (both between Iowa City and Prague and between Prague and 
some of the school sites), as well as differences in perspectives between members of the 
project team. There were insufficient opportunity to monitor parts of the data collection 
such as interviews. Although training was provided to those who were interviewing, once 
the heavy flow of the paper rating sheets began they tended to overwhelm all other methods 
of data collection. 
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Relating This Project to Other Projects in Post-Communist Countries 

Kevin Quigley 6 , a former foundation program officer, in a recent review of foundation 
funding to assist the building of democracy in Central Europe has identified fourteen 
priorities upon which such assistance has been focussed. Civic education ranks fourteenth 
out of fifteen and pre-university education fifteenth. Higher education has been the first 
priority receiving more than three times as much as any other priority. Nevertheless, he 
notes that several program of civic education have received funding and have prospered. 
In many cases they suffer from the same shortcomings he identified more generally, 
including lack of "convergence between the goal, assisting democracy and the means chosen 
(in particular, too extensive reliance on Western partners for program design and 
implementation)" (Quigley, 1997, p. 108). 

He further concluded: 

"Foundations' efforts to assist democracy in the region could have benefitted by 
significant local involvement in all aspects of their work, from design to 
implementation to evaluation... Foundations' efforts to strengthen democratic 
tendencies and encourage democratic values, such as participation, openness, and 
accountability could have been more effective had they more self-consciously 
modeled the behaviors they advocated" (Quigley, 1997, p. 109). 

Further, he noted that foundations are often biased toward the version of democracy in their 
own national setting and seek to create the rest of the world in that image. Since most 
foundations providing this funding in the early 1990s were American, he argued that this 
usually meant in the American image, and failed to take into account the definitions and the 
local context. He also faulted foundations for paying insufficient attention to evaluation and 
for failing to involve their local partners in a significant way in the evaluations that were 
implemented. 

He also spoke about the importance of having realistic goals, estimating that many of the 
stated goals for democratic reform in Eastern and Central Europe are likely to take at least 
two generations to accomplish. Setting goals that are more modest than full-scale 
educational or political reform would have provided clearer benchmarks for the evaluation 
of foundations' programs. Finally, he argued for the importance of empowering local 
partners and engaging them at every step as essential for sustainability. 

Concluding Comments 

Taking these general comments based on Kevin Quigley's analysis of foundation 
programming early in the 1 990's, has provided a framework for judging the contributions 



6 Kevin F. F. Quigley (1997). For Democracy’s Sake: Foundations and Democracy Assistance 
in Central Europe. Baltimore: Johns Hopkins University Press. 
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made by the University of Iowa/Czech Republic project. This project was funded by a U. S. 
federal agency (not a foundation) and its implementation and evaluation were planned before 
Quigley's book was available. Nevertheless the designers of this project made substantial 
steps in the directions that were recommended in For Democracy ’s Sake. 

The experience of the Civic Education in the Czech Republic Project effectively built 
lessons, training, and teacher commitment on the experience of previous projects in this part 
of Europe. There was definitely an effort in this project to take the Czech context into 
account, presenting a variety of models and resources and giving free reign to the 
professional educators to adapt them for Czech use. There was also an effort, largely 
successful, to make the project a model of openness and democratic participation. The 
project paid considerable attention to evaluation and involved the Czech participants in that 
process. The goals were, as in many projects, very high (see previous discussion of the 
tradeoff implied in choosing less lofty goals). Finally, local partners were clearly 
empowered by the process. Both the results (in the form of lessons) and the preparation 
received by the Czech educators to implement and further improve them in the future can 
be judged as very successful. 
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APPENDIX 1 
Judith Torney-Purta 



Lesson Topics 

Topic/sub-topics 

Goals 

1. Democracy 
.concept of 
democracy 
.citizen in 
democracy 
.values of 
democracy 



2. Ideology 
.shared ideas 
and principles 
.concept of 
religious and 
political ideo- 
logy 



Type of Student Participation. Teacher and Student Feedback 
Student Participation Feedback/Comments 



Group work Ii added discussion about 

Use text/tables school rules; queries fit 

Give reasons to curriculum 

S: learned contrast of 
democracy/non-democracy and 
limits of democracy; learned from 
cooperation in groups; want more 
discussion and more modern topics 
Ii left out some text; students 
realize a different civic and state 
relationship within democracy; 
goals clear 

S: learned about citizens' rights; 
want more suitable illustrations 
Ii left out role playing substituting 
discussion about smoking rights; 
strength in relating traditional and 
contemporary values; shows necessity 
of encountering different opinions in 
democratic society 
S: learned from research and 
discussion; did not understand 
aggressive foreign politics 
want more discussion, less articles 



Search in dictionary Tj. left out some text and graphs; 

Discussion added current newspaper articles and 

political party programs; students 
took main role with teacher organizer; 
knowledge may be more sustainable 
since emotions were included in 
learning; maybe less information 
than in a classical lesson, but a 
a better quality 

S: learned about political party 
orientation; about discussing and 
tolerating others' opinions; saw 
hidden symbols in film; found Animal 
Farm very truthful; want better 
explanations in textbooks and more 
examples from real life 
Ii gave more examples; not 
too much material 
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S: learned what is conservative, 
socialist; learned that everybody 
has own opinion; want more material 
and practical examples, slower 
explanations 



3. Politics Group work 

.role of politics on definitions 

in society Work with schemes 

.politics in the 
life of society 
.see political 
forces operating 
in society and in 
political problems 



lileft out 1 article and politics in 
life of society; added definition of 
politics (with homework in advance); 
good in forming students' concept 
of politics and relating to problems; 
not enough time to review broad topic 
S: like communication style of lesson 
important to learn how to assert 
opinions, cannot learn that in purely 
instructional lesson, but could be 
more fun 

S: liked activities; did not 
understand why there is so much hatred 
among some political parties 
li too much material to complete 
S: learned from my own thinking; 
learned from independent work 



4. The nation and Pairwork 

the state in Czech. Individual work 

.T.G Masaryk 
.Czech state and 
patriotism 
.totalitarian state 
in Stalinist period 



li left out totalitarian state and 
added more historical background and 
d i f 1 f erent man i f estat i ons/patr i ot i sm ; 
liked lively reworking of history and 
that students came to definition of 
patriotism on their own. 

S: (all) learned from discussion 



5. Tolerance Role play 

.difference between Individual research 
tolerance and intol. 

.analyze typical 
conflicts and causes 



li students refused role-play 
added discussion of racism, 
gypsies 

S: learned what toleration means 



6. Human rights Group work 
.documents defining Individual work 
human rights 
and concepts 
.were human rights 
respected in our 
state? 

.examples of civic 
bravery 



li left out historical evolution and 
added discussion comparing respect for 
human rights in past regime and today 
S: made knowledge about h.r. more deep 
S: learned about human rights 
before and after the revo- 
lution 

li added explanations; liked the way 
practical tasks supported previous 
more theoretical knowledge; students 
by determining which right is most 
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important understood interconnections 
and importance of rights for freedom 
of man; students may get less 
amount of knowledge but those facts 
that are gained are deeper and more 
pertinent because they are based on 
own experience and emotion; could 
prepare a video with source documents 
and times when h.r. were denied (from 
history of U.S., USSR, Czech Republic, 
Germany and international trials) 

S: learned to work collectively in a 
group and that all rights are 
interconnected; use this style of 
lesson more often; learned 
fast decision making and 
to listen to opinion of others 



7. Constitution and Pairwork 
rights and obli- Group work 

gations of citizens Individual work 

.define constitu- 
tion and show its 
role 

.relations of 
constitution/laws 
.significance of 
elections in demo- 
cracy 

.role of citizens 
and media in 
elections 

.distinguish impor- 
tant from unimpor- 
tant facts 



Ii left out details in Constit. 
comparison and lessons merged; 
table on powers of different 
branches was added; found it 
a very good lesson, esp. 
relation between rights/duties 
S: too much information, want more 
about actual laws 

Ii added motto, "freedom ends where 
rights of other person begin.; lessons 
merged; lesson worked out very 
well in practice except for groupwork; 
students did express opinions; could 
add statistical information 
S: want more visual aids or 
statistics 



Ii added chart about 
election and local repre- 
sentation; students 
formed own meaning of 
importance of elections 
by participating in one 
S: learned about system 
and how to vote 
Ii students were active 
participants 
S: learned from free 
discussion with teachers 
and school mates; more visual 
aids would help 
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8. Elections Board game 

.elections in Work with text 

democracy 
.citizens' respon- 
sibility in elections 
.function of free 
media in elections 
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9. President and Work with supplement 

government of Czech Individual work 

Republic 

.executive powers 
.relation of Presi- 
dent, government. 

Pari i ament 



10. Parliament and Individual work 
political parties in Simulation 
Czech Republic 

. structure/f unct i on 
.process of law en- 
actment 

.classification of 
parties 



11. Civic community Work with media 
.lobby groups, Independent study 
NGO's, interest 
groups and their 
role 

.forming civic society 



Ii added names of present 
ministers and responsibilities; 
some materials should be left 
out, but well done lesson 
S: learned specific facts 
Ii learned to analyze documents; 
some were not able to 
solve problems or relate to it 
S: learned from reading the 
Constitution about the President; 
want more dialogues between teachers 
and students on a wider set of 
questions; shorten lesson 



Ii left out history of founding 
of political parties; linked to 
other lessons; students are 
able to express ideas, understand 
role of Parliament and duties and 
distinguish between right and left 
wing parties and programmes; wants 
to leave out several parts, including 
simulation 

S: learned from discussion 



Ti added homework to name 
organizations in which students 
are interested; wrote a petition; 
strong lesson because it was 
concrete, best to this point, 
especially for older students 
S: learning about modes of protest 
or complaint about environment probl; 
rights to join organizations 
on a voluntary basis; learned 
about writing a petition 
Ii supplement was missing so 
used own town; added historical 
material and reviewed basic rights 
documents; newspapers weren't 
used; added material about 
differences in opinions among 
people; strong in relating to 
contemporary citizen in society 
S: learned about constitutional 
order and basic rights document; 
want more activities like this 
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12. Citizen and the Individual work with 
state questionnaire, essay 

.how citizen can make [analysis of 
opinions public and newspaper article: 
listen to others organize meeting] 
.participating in 
public meetings 
(communi eating) 

.civic responsibility 
in voting, being in- 
volved, leadership 
.identifying and 
solving problems 
in public and private 
.political parties and 
the citizen 



li started with knowledge test 
about Czech Republic; added map; 
the best lesson about consciousness 
of citizenship 

S: learned from test, lecture, 
dialogue with teacher; want more 
visuals 

Ti left out some essay/letter 
assignments; helpful to add 
addresses of newspapers, radio 
at national and regional levels 
S: learned how to state opinions 
and write letter to newspaper 
II left out teaching about famous 
speakers; added students' evaluation 
their presentations; activities 
were interesting for students 
S: learned from preparation, 
public presentation, opinions 
of others 

li added homework about being 
discussion moderators; made 
discussion of being a good listener 
starting point for discussing free 
speech, tolerance, civil rights; 
student self-evaluation; want more 
about comparing face-to-face 
listening with video 
S: learned to analyze my own 
qualities and listen to others; 
would help to have recording of 
discussion; would like more extracts 
Ti added material about styles of 
meeting, agenda, report; 
students recognized that these topics 
are important for civic community 
S: learned how to organize a 

public meeting, getting information 
about our community; rights to 
decide about public affairs; 
preparing to contribute to discussion; 
want visit to Parliament; want 
visit to town committees 
Tj. added newspaper article, gave more 
examples of vandalism and what 
they would do if they saw it 
S: want prices of items vandal - 
might damage 

li added analysis of newspaper 
articles: objectivity and 
verification; strength was 
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critical analysis of articles 
S: understood how newspapers 
may give subjective information; 
want more concrete materials. 

Ii strength is concreteness 
S: learned about citizens' rights 
in a concrete sense 
Ii not enough time for profiles 
and chart; strengths are 
stimulating activity of students 
and critical evaluation 
S: learned about features of 
leaders and classmates' opinions 
Ii establishing a civic association 
might be better than a party 
S: learned how to establish a 
party, but add more about comparing 
party programs before voting and 
chart about where parties of each 
country stand 

Ii added points about process of 
problem solving to chart and dis- 
cussion about community responsibility 
S: learned how to face up to 
problems from practical examples; add 
more discussion 
Ii no weaknesses, strength in 
critical point of view 
S: learned about characteristic of 
president; want more pictures and 
real examples 

Ii applied material to improving 
relationships in the class which 
was a strength for students' awareness 
S: learned about improving rela- 
tionships and role of individual 



13. Municipality Dialogue teacher/ 

and individual student 

.difference of Prepare written 

state/mun i c i pa 1 . mater i a 1 

.how citizen can 
influence and solve 
problems in municip. 



Ii added examples from everyday 
life; deepened students knowledge 
about responsibility and work of 
authorities and civil laws; students 
may get less quantity of knowledge, 
but what they acquire is deeper and 
will last longer because they made 
made their own conclusions 
S: learned today by listening to 
others' opinions and finding 
compromises in solving problems; 
add more work in small groups and 
practical activities to solve 
concrete problems. 
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14. Law awareness Simulation 

.legal system Discussion 

.personal rights and 
statutory respon- 
sibilities 

.issue of death penalty 



15. Legal code Discussion 

.what belongs Question answer method 

to legal code 
.how law is 
enacted or 
changed 



16. Free market Analysis of text 

economy Interview 

.basic economic 
thinking 

.production, con- 
sumption, distri- 
bution 



li role of newspapers is dif- 
ficult for vocational students; 
divided class into groups of 6; 
strength in collaboration; add 
vocabulary list 

S: learned by working in groups; 
did not understand censorship or 
petition 

II capital punishment better 
integrated into lesson; strength 
was discussion and interest 
S: learned importance of law 
for society; learned from discussion; 
want more examples 
II founds parts of topic contro- 
versial and left out; added pupil 
questionnaire about death penalty; 
students learned from discussion; 
did not understand some terms 
S: learned from articles and 
group work that life is unique 
and about guilt and innocence; want 
more examples 



li left out "rule of law;" 
simplified the chart; strong groups 
S: learned from self-study of chart 
and handouts and from discussion 
li left out how Parliament works and 
added material about rule of law 
S: learned from handouts and groupwork 



li added handout about market; 
students understood what influence 
differences in human needs; too many 
new terms 

S: learned about competing among 
students; want more discussion about 
problems of young people 
li added comparison of local, 
national, international market 
and those subject to market 
(enterprises, households, state); 
strength in distinguishing among 
human needs; too many new concepts 
S: learned from communication and 
comparison with classmates; school 
should deal more with problems of 
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17. Economy and 
ways to prosperity 
.function of money 
.banking 

.role of state in 
economy 



young people and compare with adults 
(drugs, money, social issues) 

Ii strength in understanding 
production, consumption, distribution, 
also supply and demand; add more 
about free market mechanism 
S: learned about free market and 
private business; how to prosper 
and come to the top and not bankrupt; 
learned from discussion; want 
better explanations and more examples 
Ii added examples of central planning; 
strong in basic concepts but need 
better examples 

S: learned about economic systems; to 
divide economic spheres; discussion 
was interesting 



Analysis of text Tl goals were fulfilled; need 

Discussion more examples to illustrate 

function of currency 
S: learned about function and 
value of currency; overtaxed my 
brain by making it really think; 
it was an exceptional lesson in which 
I proved to myself that I know 
something about economy; our younger 
classmates did not understand 
Ii added material about banks; 
such as relation to foreign banks 
and difference between cash/non-cash; 
too much to explain; hard to under- 
stand recession; need more examples 
from practical life 
S: learned what to beware of in 
banks, basic services and 
financial reserves; didn't understand 
recession or inflation; want speaker 
who works in a bank; too much subject 
matter; had to think a lot. 

Ii added intervention of state 
into economy and newspaper article; 
students discussed a variety of 
economic problems; some students 
made negative statements about economy 
S: learned about state's task in 
economy; learned from exchange 
of opinions want more examples, video 



18. Problems of 



Group work 



Ii added overpopulation, hunger. 



contemporary Audiovisuals 

world 
.AIDS 

.drugs/alcohol 
.violence and 
terrorism 



19. Ecology, Pairwork 

economy, and Work with sup- 

politics plements 

.how students 
can contribute 
to solve prob- 
lems of environ- 
ment 



countries in debt; 
students defined problems and 
attempted solving some of them 
S: learned about problems that 
involve our lives; don't under- 
why we don't fight more against 
these problems; add discussion 
with persons or organizations 
with responsibilities for such 
situations 

Ii added photographs; should add 
a suitable video 
S: statistics were most useful, 
however, many knew about drugs 
already. 

Ii students should not close 
their eyes to these problems 
S: lesson was clear and understood 
Ii students learned about causes, 
roots, consequences, and protection 
against violence; could use video 
S: didn't understand what our 
state does to protect against 
violence and why people are passive; 
want video and real examples 



Ii added material from a 
textbook 

S: learned that there are 
problems of ecology relating 
to economy and politics 
Ii added concrete figures 
about our region and national 
parks, also material from a 
textbook; good lesson for 
its concreteness 
S: learned from newspapers 
and statistics 



20. World com- Work with maps and 

munity and nation tables 

. i nternat i ona 1 Groupwork 

organizations like 
UN and NATO 
.deepen independent 
thinking 

.appreciate the im- 
portance of organizations 



Ii left out NATO; added a 
chart about international 
organizations; want more 
charts and graphs, clearer 
instructions needed for some 
classes and ways to make 
more interesting with examples 
S: it would be more interesting 
to analyze newspaper articles 
and to have a more intelligible 
text 
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li added foreigners' views on 
CR (material on sport, 
art, music, inventions); 
strength in supporting civic pride 
and realizing our priorities 
S: realized importance of 
pride and responsibility of CR in 
the world 

li added chart about international 
organizations and speech by Havel 
S: learned about world communities, 
internet, NATO and its origins 



21. European Groupwork 

Community Text analysis 

.EU and cooperation 
.CR and the EU 
.need for personal 
involvement in links 
in Europe 



li students realized what is 
necessary to do for European 
integration; many students 
didn't have facts about EU 
S: learned from discussion 
and writing an interpretation 
of history and objectives of 
EU; want more charts; didn't 
understand embargoes 
li didn't choose a land for travel 
and didn't simulate a parlia- 
ment; strength that students had 
time to discuss; students didn't 
understand duties to EU 
S: understood values of CR and EU 
authorities; learned from discussion 
of parties' positions; didn't 
understand arguments against 
entrance to EU; didn't understand 
European Parliament; want a video, 
more examples and material about 
advantages and disadvantages 



APPENDIX 2 



Barbara Malak-Minkiewicz 



Some notes on interviewing teachers 

(Prepared for the Czech - American Project on Civic Education) 

February 1997 



Type of the interview and its characteristics 

The main goal of the interview in this project is to collect in-depth information from the 
pilot teachers who tested civics lesson scenarios. The content of the interview should be 
developed on the base of the data included in the short feedback questionnaire filled in after 
each lesson by both teacher and students. The researchers have to learn in detail what 
worked/didn t work, why and possibly — how the lesson could be changed/improved. The 
subject of the interview and characteristic of the interviewee decide about how it should be 
prepared and conducted. 

Two aspects of interviewing should be taken into consideration here: the extent to which the 
interview is supposed to be standardized, and the extend to which interviewees will be 
willing to disclose the information. 

(i) In the fully standardized interview, the investigator defines the questions and the 
problems. He/she is looking for answers within a frame set by his/her presuppositions. 
On the other extreme there is a fully non-standardized interview: the interviewee serves 
as an ‘expert’ who teaches the interviewer what the problem, the question, the situation 
is. The type of interview that seems to be most suitable for the project under 
consideration (collecting evaluative data form the teachers who tested in the classroom 
practice lesson scenarios) is located somewhere in between these two extremes: 
problems and issues are to some extend defined in advance but there are pilot teachers 
who are experts on evaluation of the specific lesson scenarios they tested. This is the so 
called guided or focused interview (there are selected topics around which the 
interview is carried out). 

(ii) Although it can be assumed, that since a person agreed to be interviewed, he/she wants 
to disclose information he/she possess, this is not always the case. For example, 
politicians usually want to be interviewed but they do not want to make public 
‘everything’ they know on the subject. There is not such a problem in the case of the 
interviews under discussion: from definition the interviewer and the interviewee have 
the same goals to find out the strong and week points of the lesson scenarios. Thus the 
main task of the interviewer is to help the pilot teacher to articulate his/her opinions. 
There is one problem however that makes this common task ‘not common’ in one 
respect: the teacher may want to hide from the researcher any information that he/she 
considers might suggest his/her professional shortages. The failure or success of a 
lesson is always the result of program-teacher interaction and of course each teacher 
knows this. This means that the teacher may try to down play what he/she thinks (even 
if inadequately) was his/her personal “fault”. On the other hand — this information may 
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be very important for the researcher because even the ‘very best’ lesson scenario is not 
good at all if it does not fit possibilities of many teachers. 

The interviewing teacher as an expert 

Three important characteristics of expert in the context of a interview have to do with: 
his/her status and competencies, lack of anonymity, and proficiency in being 
interviewed. 

(i) ‘Experts usually feel and behave like experts’ means that very often they want to take 
over the leadership from the interviewer. They want to define a situation for themselves, 
explain how they see it, what the real important problems are as they view the matter. In 
the case of teachers however (and for sure in the post-communist part of Europe) this 
issue is more complicated. On the one hand — teachers are considered to be 
(indisputable) experts for students and (often) for their parents. On the other - they are 
rarely treated as experts by educational policy makers, curriculum developers and/or 
authors of textbooks. They feel very often “underestimated”, frustrated, not recognized 
with their expertise rooted in educational practice. Keeping this in mind, if researchers 
in this project want interviewed teachers to feel and behave as experts, they need to 
assure them that they are experts , that their competencies are highly appreciated. 

(ii) It is usually difficult to assure anonymity of an expert, e.g. some of them are broadly 
known and their opinions are broadly known. It requires thus special measures to 
protect such persons’ identity from being disclosed to third parties. In the case of the 
project under consideration anonymity is also an issue: there is a small and visible group 
of pilot teachers being interviewed and they have various relations with the authors of 
the project. It may produce a feeling of “uneasiness” when expressing negative opinions 
about tested material. 

(iii) Experts are most often proficient in being interviewed and sometimes they may no 
longer reply to interviewer questions with information from experiences, but with 
answers, which they learn as proper from previous interviews (and interviewers). A 
majority of teachers do not have such experiences. They have however a very specific 
sense of the ‘propemess’ of answers (originated from the student-teacher relation) and 
may report not what they ‘think about the issue’ but ‘what they think they should think 
about the issue’. On the other hand - teachers ‘want to teach’: they are often ‘talkative’ 
persons, they are sensitive on being listened to and understood. This is an advantage. 

Some suggestions for getting, conducting and recording interviews 

The most universal rule for specialized interviewing is that the best way to interview in a 
concrete situation depends upon that situation (including the skills and personalities of the 
interviewer). But in any case, in advance of each interview some aspects of it should be 
carefully thought through. The most important are: 

0) What sort of relationship should the interviewer establish with the interviewee ? Many 
experts in the field argue that the interviewer should try to get cooperation by 
deliberately seeking to establish ‘neutrality on the interviewers side’: sympathetic 
understanding, willingness ‘to talk informant’s language’. In some specific situations 
this technique doesn’t work: sometimes interviewer can not identify him/herself with 
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the interviewee’s values and opinions, sometimes - the only result of the interviewer’s 
support is a not very useful type of informant’s behavior: ' everything is just great, 
material is great, lessons are great, students are great, I am great'. In such 
circumstances ‘neutrality against the interviewee’ can be adopted (which may even 
lead to a kind of dispute). This more assertive technique is sometimes useful in 
interviewing teachers (who adopt sometimes a behavioral pattern of the ‘best student in 
the classroom’, who wants to please the interviewer). It is not recommended however 
for persons who are not very much experienced in interviewing. A more careful 
approach is to be advised instead: ‘conscious role playing’ (e.g. the teacher might be 
asked about possible problems with the lesson scenario which would be faced by 
somebody who is much less experienced than he/she; the interviewer may ask questions 
from the position of the ‘devil’s advocate’ at the same time expressing his/her personal 
agreement with the interviewee’s opinions). 

(ii) What does the interviewee get out of the interview ? Probably, the greatest value which 
many interviewees receive - the reason why they enjoy the interview - is the opportunity 
to teach. This is probably why a simple ' teach me 9 approach of the interviewer seems to 
be more useful than others. Teaching provides the informant with a chance to articulate 
his/her ideas and to present the importance of his/her job. The simple opportunity to talk 
to somebody who understands, whose comments are relevant, but who will not make 
any further claims ( an ‘understanding stranger’) can also provide a pleasant experience. 
All of this makes teachers easier than any other group to be interviewed. One should 
stress however that the ' teach me ' approach is something else that the ( tell me 
everything ’ approach. This latter method can lead to the collection of a large amount of 
possibly interesting but not always relevant information. In addition, some more ‘task 
oriented’ interviewees get angry with the vague questions about ‘everything’ as if 
interviewer did not know his/her job. 

It happens sometimes that researchers are confronted with expressed or implied request 
by the interviewee for a ‘small token of appreciation’ (e.g. good opinion about 
informant expressed in the front of the supervisor). Besides the ethical ambiguity of 
such procedures one should be aware of the danger it may bring for the project and 
researchers themselves. It can involve them into interpersonal and political struggle and 
conflicts. 

(iii) How to introduce the interview ? For any interview an informed commitment of the 
interviewee is required. It is therefore important that that the introduction is clear and 
does not bear any ambiguity. It is usually recommended that the introductory 
description of the interview’s purpose should not be too complex and should not include 
too much details. This is first, because the more complex the information is, the higher 
the possibility is that the interviewee will doubt whether he/she will be able to perform 
well (and may feel embarrassed and tense). Too much information may also bias the 
course of the interview: the interviewer can not predict all possible ‘hidden 
assumptions’ about the situation that interviewee can produce. The explanation must be 
couched in more general terms and should include the same information for all 
interviewees (such a standard introduction could be developed by researchers in 
advance). Along with a description of the task it should include clear information on 
why and how the interview results will be used. It is also recommendable to check at the 
beginning of the interview what the interviewee thinks about the purpose and potential 
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usefulness of such an interview (it helps the researcher to leam and to eliminate some 
interviewee’s ‘hidden assumptions’). 

0 V ) How and where to arrange t h e interview ? The general advice is that the interviewer has 
to adjust to the interviewee preferences. One should accept any time he/she proposes, if 
at all possible. The interviewee should also feel free in choosing the place for the 
interview, as long as interviewer is sure that this place is quiet and gives privacy. This is 
important because when other people (especially colleagues) are watching the person, 
he/she is likely to be more careful about his/her actions. Public places such as 
restaurants, coffee shops, etc. are not recommended: they are noisy, one can meet 
colleagues there who interrupt the interview, etc. Arranging the interview means also to 
establish how much time is needed for it. This should be negotiated with the interviewee 
but when established (and especially when time limits are fixed) should not be 
exceeded, if possible. 



( v ) How to present yourself? It is important to explain at the beginning of the interview who 
you are but it is not easy to tell what information should be included. The interviewee 
should be assured that you are a qualified person to conduct this interview and that you 
can be trusted (not having any ‘hidden agenda’). Thus you should mention your 
professional competencies and your professional interest in the project. One should 
avoid however information which could make the interviewee think that interviewer’s 
expertise is so high’ that he/she rather wants to examine the informant knowledge 
rather than to leam something. One also should avoid too much personal information. 
On the one hand it helps sometimes ‘to break the ice’, on the other - may not be 
welcomed by the interviewee who does not want to disclose any personal information 
by him/herself. In any circumstances the best credentials for the interviewer are those 
which show quite clearly that one is whom one says to be and that the project in which 
one is engaged is bona fide. 

( vi ) How to start and proce ed with the interview ? The interview should start with general 
question which can be interpreted by the informant in several different ways and out of 
his/her own experiences. This gives the interviewee from the very beginning great 
freedom of expressing him/herself without an embarrassing feeling of ‘not knowing the 
answer’. 

The whole interview should be organized in the framework of a few such general 
questions. They should be the same for all pilot teachers while more detailed questions 
will be specific for each given interview and dependent on its development. 

It is recommended that a group of researchers collaborates in the preparation of the 
interview plan in advance. Feedback sheets filled out by teachers and students based on 
experiences with lessons can be of a great help for this. 



( vn ) Taking notes, using ta pe recorder . There is wide variation between interviewers as to 
how to take notes. Some believe no notes should be taken during the interview but the 
majority is just of the opposite opinion. Everybody agrees that the interview should be 
written down as soon as possible after being completed. To take notes and not loosing 
contact with the interviewee is especially difficult for the less experienced interviewer. 
In such a case a kind of standardized report is recommended to be prepared in advance 
(or at least a list of headings under which responses can be recorded). Tape recording 
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gives accuracy but it also has its shortages: transcription is time-consuming and some 
interviewees do not like to be recorded. It should be carefully considered therefore, 
especially in post-communist countries, where people still have many fears due to their 
past experiences. 

(viii) Protecting the confidentiality of the interview . The interviewee should always receive 
assurance that the interviewer himself intends to maintain discretion but also that he has 
adequate control of the reports or tapes after the interview is completed. The number of 
people who have access to interviews should be limited to those who really have ‘a need 
to know’. It must be assured that when data leaves the interviewer’s hands to the project 
headquarters office, great caution is exercised there. 

Good interviewee and good interviewer 

In any interview that interviewee’s statements represent nothing more than his perception, 
filtered and modified by his cognitive and emotional reactions and reported through his 
personal verbal usage. The interviewer must develop self-consciousness about what is 
affecting the interviewee, including how he/she himself affects the person. 

(i) Sincerity and reliability of the interviewee . The data we collect through interview are 
never fully consistent. Putting different kind of questions in different ways the 
interviewer elicits different kinds of ‘subjective data’: on emotional states, values, 
attitudes, etc. The interviewee may have a positive opinion about something in one 
context and a negative one in another, and his observations are by their very nature 
selective. This selectiveness carries however important information and should not be 
interpreted as a lack of sincerity of the informant. There are some factors however 
which may influence his/her reliability and the key question for evaluating data is: what 
are these factors in given circumstances? The following are likely to be important: 
ulterior motives, bars to spontaneity, desires to please interviewer, idiosyncratic 
factors. They could be diminished when the interview is conducted in a proper way: in 
a friendly, good atmosphere, and with full assurance of confidentiality. This is also very 
important for the interviewee to believe that interviewer’s role is of an ‘objective 
researcher’: that he/she does not have any ‘hidden’ purposes and is not in the position to 
influence the situation. Idiosyncratic factors of connotations and meaning are very 
difficult to account for. The problem that may arise in the context of the interview under 
discussion is that some questions may be perceived as enforcing specific (‘good’ for the 
interviewer) answers. Some of them could also make the interviewee feel like being 
examined. A good precaution is to ask questions in many different ways. 

(ii) What makes a good interviewer ? Three characteristics of the interviewer are of 
particular importance: to be a good listener, to have a skill of empathetic 
understanding, and to have the ability to control the interview. 

Good attention of the interviewer means efficient and concentrated attention. It 
happens that the interviewer hears some isolated point and then, instead of listening to 
the following sentences, he/she proceeds by building upon this point some notion of 
his/her own, instead of on what the interviewee is trying to say. It is very important 
therefore to follow carefully and to full extend what speaker is saying, because the 
assumption that ‘7 can guess what he/she is going to say' may be very incorrect. 
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Usually the interviewer has his/her own opinions and hypothesis about the subject of the 
interview. Researchers may like some lesson scenarios more than others or may have 
their opinions on what may be difficult for teachers, interesting (or not) for students. 
This produces a tendency to test one’s own hypothesis and to omit information that 
seems irrelevant to them. To avoid this, the interviewer has to concentrate fully on the 
collected field data (on what interviewee says). 

Empathetic understanding means the ability to grasp what the other person is 
experiencing, to feel some of what he/she feels, to share to same extent his/her view of 
his/her experiences. Though sometimes seems difficult, it is possible to put oneself in 
other person shoes’. Careful observation of the interviewee non-verbal behavior such as 
gesticulation and facial expression helps to understand better what he/she feels and 
experiences. It is important not only for the general atmosphere of the interview. 
Without such a ability the interviewer is not able to ask proper questions at the proper 
moment. 

To control a guided (focused) interview is more difficult than to control an interview 
which is more formalized. The interview with a vague structure is not a simple question- 
answer interaction. It sounds like discussion or rather a quasi-monologue stimulated by 
specific questions and understanding comments. It gives the interviewee the possibility 
to select issues for the discussion. It is the interviewer’s responsibility therefore to keg p 
informant’s speaking relevant . Each interviewee has his/her ‘own story’ that may be 
interesting but this interview has its own subject and purpose to be fulfilled. Intellectual 
flexibility and good understanding of ‘what is happening’ during the interview allow the 
interviewer to differentiate between ‘unexpected’ but relevant and ‘fascinating’ but 
related to ‘another story’ information. 

To exercise control over the interview requires also a self-consciousness about what is 
affecting the interviewee, including the interviewer as a person. Whether the researcher 
wishes or not, interviewing is a social relationship and he/she is a part of this 
relationship. His/her definitions of the situation and biases are its very important 
element. It is very difficult to see how these researcher related ‘subjective factors’ can 
be avoided completely, but awareness of the problem plus constant self-control can 
help. 
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